Use upstream Rocky Linux base images for rocky driver containers - #880
Open
abrarshivani wants to merge 1 commit into
Open
Use upstream Rocky Linux base images for rocky driver containers#880abrarshivani wants to merge 1 commit into
abrarshivani wants to merge 1 commit into
Conversation
The rocky8/9/10 driver images were built on nvcr.io/nvidia/cuda:*-base- rockylinux*, which coupled Rocky driver builds to the CUDA image release cadence. That coupling is already biting: no 13.2.1 or 13.3.0 base-rockylinux10 tag was ever published, which is why rocky10 was pinned to 13.2.0 while rocky8/9 moved to 13.3.0. Point the rocky targets at Rocky Linux's own images instead, mirroring how the rhel targets build on registry.access.redhat.com/ubi*. The -ubi flavor is used because its package set matches the Red Hat UBI images that the rhel Dockerfiles are written against - notably it ships `which` and systemd, which the default Rocky image does not. This drops nothing the driver container used. The CUDA base only added cuda-cudart, cuda-compat and the cuda-toolkit-*-config-common packages on top of the plain Rocky image; none are referenced by the rhel Dockerfiles, which do not even declare ARG CUDA_VERSION. The CUDA package repository that supplies fabricmanager, nscq, nvsdm and imex is unaffected: it is added explicitly by install.sh setup_cuda_repo at build time and is independent of the base image. The base image's own cuda.repo was already being deleted before the final layer. Renovate needs new guards because all three Rocky majors share a single image repository, so an unconstrained bump could move rocky9 onto a Rocky 10 tag. The previous nvcr.io tags were implicitly protected by docker versioning's suffix compatibility. Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
Contributor
|
Thanks @abrarshivani! The changes look good to me. Can we paste the logs of running driver containers on RockyLinux systems built from these changes? Once done, these should be good to merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind cleanup
What this PR does / why we need it:
The
rocky8/rocky9/rocky10driver images are built onnvcr.io/nvidia/cuda:*-base-rockylinux*— a CUDA runtime image used only as "a Rocky image on a registry we control". This points them at Rocky Linux's own images instead, matching how the rhel targets build on Red Hat UBI.The current setup ties Rocky driver builds to the CUDA image release cadence, and that is already causing drift:
13.2.1-base-rockylinux10and13.3.0-base-rockylinux10were never published, which is whyrocky10is pinned to13.2.0whilerocky8/rocky9are on13.3.0. If therockylinuxCUDA variants are ever dropped, Rocky builds break outright.This also adds Renovate guards. All three Rocky majors share a single image repository, so an unconstrained bump could move
rocky9onto a Rocky 10 tag — the oldnvcr.iotags were implicitly protected by docker versioning's suffix compatibility.Which issue(s) this PR fixes:
Fixes #764
Special notes for your reviewer:
Why the
-ubiflavor. The rhel Dockerfiles are written against Red Hat UBI, and Rocky's-ubiflavor is built to match the UBI package manifest, so the shared Dockerfile works unmodified. The default Rocky flavor lackswhich(used byocp_dtk_entrypoint) andsystemd, both of which UBI and-ubiship.Nothing the driver container used is lost. Diffing the RPM databases,
cuda:13.3.0-base-rockylinux9is exactlyrockylinux/rockylinux:9pluscuda-cudart,cuda-compat, threecuda-toolkit-*-config-common,libxcrypt-compatand agpg-pubkey. None are referenced anywhere underrhel{8,9,10}/— those Dockerfiles do not even declareARG CUDA_VERSION, so the--build-arg CUDA_VERSIONthe Makefile passes is already discarded as unused.The CUDA package repo is untouched.
install.sh setup_cuda_repoadds it from a hardcodeddeveloper.download.nvidia.comURL at build time, independent of the base image — which is why the rhel builds already work on plain UBI with no CUDA base anywhere. The base image's owncuda.repowas already deleted before the final layer.install.shalso already handlesID=rocky→ enablecrb, anddnf config-managerarrives as a weak dep ofepel-releasein both the old and new base. Theprecompiledpaths genuinely need CUDA images and are not touched.What I verified.
make -nresolves all three rocky targets to the new bases, andrhel8/rhel9/rhel10still pass no override and keep their UBI defaults. All three pinned tags exist on Docker Hub withamd64andarm64.renovate.jsonis well-formed.What I could not verify — please
/ok to test. Draft because the images have not been built; no container runtime was available to me. Theimageworkflow building rocky8/9/10 across both driver versions and arches is the real gate.On registries. This uses the public
rockylinux/rockylinuxreference rather than an internal mirror.dockerhub.nvidia.comresolves only on the NVIDIA network, so hardcoding it would break external contributors, customers rebuilding driver images from source, and Renovate. Docker Hub is already a build dependency here via theubuntu:*bases (anddocker:dindin GitLab CI), so this adds no new registry. If internal pull-through caching is wanted it belongs in the runners'buildkitd.tomlas adocker.iomirror, where it would transparently cover the Ubuntu pulls too.Does this PR introduce a user-facing change?